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DETAILED ACTION 



1. This Office action for U.S. Patent Application 10/713,130 is responsive to the 
Request for Continued Examination filed 14 October 2009, in reply to the Final 
Rejection of 17 July 2009 and the Advisory Action of 23 September 2009. Currently, 
Claims 11, 12, 14-17, 19-24, 35, 49-53, 55-58, 60-65, and 67-70 are pending. 

2. In the 17 July 2009 Office action. Claims 52, 53, 55-58, 62, 63, 67, 68, and 70 
were rejected under 35 U.S.C. 101 as non-statutory. Claims 11, 12, 14-17, 19, 20, 23, 
24, 35, 49-53, 55-58, 60, 61, 64, 65, and 67-70 were rejected under 35 U.S.C. 103(a) 
as obvious over U.S. Patent 6,111,604 A (Hashimoto) in view of U.S. Patent 6,148,031 
A {Kato), U.S. Patent 5,987,179 A {Riek), and ISO/IEC 11172-1 {MPEG-1 Part 1). 
Claims 21, 22, 62, and 63 were rejected under 35 U.S.C. 103(a) as obvious over 
Hashimoto, Kato, Riek, and MPEG-1, and in view of U.S. Patent 6,327,423 B1 {Ejima). 
In the 23 September 2003 Advisory Action, the claim rejections under 35 U.S.C. 101 
were withdrawn. 



Continued Examination Under 37 CFR 1.114 

3. A request for continued examination under 37 CFR 1.114, including the fee set 
forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this 
application is eligible for continued examination under 37 CFR 1.114, and the fee set 
forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action 
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has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 14 
October 2009 has been entered. 



Response to Arguments 

4. Applicant's arguments with respect to claim 11 have been considered but are 
moot in view of the new ground(s) of rejection. Applicant has perfected foreign priority 
to Japanese application 98-00629303, filed 24 August 1998. This foreign priority 
document antedates Wilt, "The DV, DVCAM, & DVCPRO Formats", cited in the 
Advisory Action in support of the official notice statement of the Final Rejection. 
However, European Patent Application Publication 843,470 A1 (Tanaka), published 20 
May 1998, also discloses locked audio. 

5. Applicant's arguments with respect to claim 1 1 have been fully considered but 
they are unpersuasive. Applicant states that "nothing in the prior art shows, teaches, or 
suggests an apparatus or method encoding still picture data without audio data and a 
second encoding method encoding audio data with moving picture data or still picture 
data". However, as mentioned in the 17 July Final Rejection, the Hashimoto reference 
discloses producing an association file that may link image and audio files together. 
Final Rejection, page 6. The file may indicate information of "just a still image", or "still 
picture data without audio data" as claimed. Hashimoto, column 10: lines 1-2. The file 
may also store information of "the combination of a still image with audio data", or "still 
picture with audio data", as claimed. Id. at column 10: line 2. Hashimoto may also store 
information of "a series of images" or "moving images" in an MPEG format. Id. at 
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column 10: lines 3-4; column 6: line 65. MPEG was known to encode image data with 
multiplexed audio data. MPEG-1, §1-A.6.3. Thus, Hashimoto discloses all of the 
methods of encoding still picture data without audio, encoding audio data with moving 
picture data, or encoding audio data with still picture data. In the combination with the 
Kato and Riek references, the Hashimoto reference is modified to use a unified coding 
method for encoding audio data with moving picture data or still picture data. 



Claim Rejections - 35 USC § 103 

6. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth In section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

7. Claims 11, 12, 14-17, 19, 20, 23, 24, 35, 49-53, 55-58, 60, 61, 64, 65, and 67- 
70 are rejected under 35 U.S.C. 103(a) as being unpatentable over US Patent 
6,1 1 1 ,604 A {Hashimoto) in view of US Patent 6,148,031 A {Kato), in view of US Patent 
5,987,179 A {Riek), and in view of ISO/IEC 11172-1 {MPEG-1 Part 1). Hashimoto 
teaches a digital camera. 

Regarding claims 11, 35, and 52, figure 8 of Hashimoto et al. shows a block 
diagram of the camera. Image photographing section 6 comprises lens 7, lens opening 

8. imaging element 9, and filter 10. The analog input image signal is converted in 
analog/digital converter 4 and further processed in DSP 11 (column 6: lines 40-61). 
This is the claimed "photographing means". Audio signals are input into microphone 1 
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and output through amplifier/filter 2a to analog/digital converter 4 (column 6: lines 18- 
26). This corresponds with the claimed "audio inputting means". Image data 
compression/expansion circuit 12 encodes the images from DSP 1 1 in a format such as 
JPEG or MPEG (column 6: line 62-column 7: line 2). This corresponds with the 
claimed "video encoding means" that performs the steps of "encoding the video signal" 
in the two encoding methods in claims 35 and 52. Figure 11 of Hashimoto et al. 
illustrates the process for capturing video and information. When the user presses the 
shutter button, a first picture with associated audio is captured. Image and audio files 
are stored in memory card 16, and a relation file is written to link the image and audio 
files together (column 9: lines 46-54). The relation file may be a container file for "just a 
still image", "a still image with audio data", or a moving image with audio data (column 
10: lines 1-8). Then, Hashimoto discloses capturing picture data both with and without 
audio data, as claimed. 

The present invention differs from Hashimoto et al. in that in the present 
invention, two encoding methods exist: a first mode encoding only still image data, and 
a second mode encoding audio data with still picture data or moving picture data 
comprising intra I frames and inter P or B frames. However, in Hashimoto et al., no 
clear distinction is made between various encoding modes for the recorded pictures. In 
the example given in column 9: line 55-column 10: line 4, audio may be associated with 
a still JPEG image, or a moving MJPEG image, or an MPEG image (column 6: line 65), 
which was known in the art at the time the invention was made to incorporate sound 
data. It appears that the list in column 10: lines 1-4 may disclose three distinct coding 
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modes: a still image without audio; a still image with audio; and a moving image, 
whereas the present invention presents a unified encoding method for still and moving 
images with audio. 

Kato teaches an image processing system in a digital camera. Regarding claims 
1 1 and 35, in Kato, in a continuous imaging mode, input images are initially recorded in 
real time in an intra mode as a succession of JPEG images, and stored in memory 20 
(column 3: lines 41-47). In a still image mode, the input image is recorded in memory 
20 as a single JPEG image (column 3: lines 47-53). After recording is finished, system 
control circuit 26 re-encodes the recorded series of intra images in an inter-frame 
compression mode (column 3: lines 54-63). This system control circuit corresponds 
with the claimed "controlling means" of claim 11, and the selection of a still image mode 
or a motion image mode in Kato corresponds with the claimed recording mode selection 
in claims 35 and 52. 

Hashimoto et al. discloses a portion of the claimed invention, but not encoding 
pictures according to two different encoding methods. Kato teaches that it was known 
to encode motion image data in a separate format than still image data. Therefore, it 
would have been obvious to one having ordinary skill in the art at the time the invention 
was made to modify the camera of Hashimoto et al. to re-encode pictures having a 
temporal aspect, such as pictures with associated sound, as inter-frame encoded 
images after encoding, as taught by Kato, since Kato states in column 2: lines 16-34 
that such a modification would enable the final recording to be achieved with higher 
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compression than witli intra pictures alone, wliile maintaining the ability for a user to 
record a high-quality still image during the motion image recording process. 

The present invention differs from the combination of Hashimoto et al. and Kato 
in that in the present invention, a still image recorded with sound is recorded in the 
same mode containing I pictures, P pictures, or B pictures as motion images recorded 
with sound, in Hashimoto et al., a still picture recorded with sound is a single JPEG 
image with an associated audio file. 

Riek et al. discloses a camera that encodes still images in an MPEG bitstream. 
Regarding claim 1 , figure 2 illustrates an embodiment of the Riek et al. apparatus. Light 
is input through lens 12 to CCD 14, which forms images (column 4: lines 15-18). These 
images are converted to a standard digital format in ISO CCIR601 converter 27 (column 
4: lines 35-38). As will be shown below, the images received may be encoded as still 
images or moving images. A user may switch from recording motion images to 
recording still images with still select button 22 which causes logic and control unit 32 to 
encode a still image (column 4: lines 41-50). During a still image mode, a still image 
stored in frame store 29 from converter 27 is selected for encoding (column 4: lines 41- 
46), rather than directly from the converter 27. Encoder 30 encodes a still image as a 
series of zero-motion-vector B frames or an enhanced I frame or P frame followed by a 
series of B frames, and encoding the first frame at the conclusion of recording the still 
image as the next I frame (column 9: line 22-column 10: line 41). These B pictures are 
encoded entirely with skipped macroblocks, which inherently have zero motion vectors, 
or macroblocks explicitly coded with zero motion vectors. Riek at column 10: lines 53- 
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65. Such a picture, comprising sl^ipped macroblocl^s, wliich are copied from a 
chronologically preceding picture, and motion-compensated with zero motion vectors, is 
the claimed "P or B picture data" with 0 motion vectors for a entire frame, copied from a 
chronologically preceding picture. Then, R\ek et al. discloses the claimed second 
encoding method which may capture both still picture data and moving picture data, 
each comprising I pictures and P pictures or B pictures. 

Hashimoto et al., in combination with Kato, discloses a majority of the claimed 
invention except for a single encoding method suitable for both still and motion image 
data. Riek et al. teaches that it was known to encode a sequence of still pictures as 
MPEG pictures integrated within a motion picture sequence. Therefore, it would have 
been obvious to one having ordinary skill in the art at the time of the invention to modify 
the encoder of Kato to record still pictures containing sound as inter pictures as with 
motion pictures containing sound, as taught by Riek et al., since Riek et al. states in 
column 3: lines 1-59 that such a modification would allow for high quality still images to 
be encoded within a motion image at a relatively low bit rate taking advantage of the 
inter-picture coding techniques of MPEG. 

However, while the present invention is directed to multiplexing an encoded 
picture signal and an encoded audio signal, Hashimoto et al. does not give details of its 
process of "combining" a video file and an audio file (column 1 1 : lines 34-42). 

MPEG-1 Part 1 defines the system coding layer of an MPEG-1 coded data 
stream, in which audio and video data streams are multiplexed (forward). Regarding 
claims 11, 35, and 52, Section 1-A.6.3 illustrates a sample multiplexing of a stream 
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having one video and one audio stream. Tine stream is divided into pacl<s, each of 
which has a header and three pacl^ets, each of 2048 bytes. Section 1-A.6.5 illustrates 
the distribution of frames into the packs, with a first pack typically encoding I, P, and B 
picture data and a second pack typically encoding P and B picture data in conventional 
GOP structure and video of average complexity. First, 13 video packets are transmitted 
to ensure successful buffering. Then, an audio packet is placed for every 6.25 video 
packets. Section 1-A.6.9 shows an extended sample multiplexed data stream. Here, a 
second audio packet is placed between the twentieth and twenty-first video packets. 
However, while in the shown example, one audio packet is placed for multiple video 
packets, the examiner takes Official Notice that it was known in the art for audio and 
video packets to be correlated in a 1 :1 ratio, as in the "locked audio" of DV, in which one 
audio pack is present for each frame. See for example European Patent Application 
Publication 843,470 A1 (Tanaka), column 2, lines 34-42. Tanaka describes a video 
signal comprising video data and audio data. In a "lock mode", a number of audio 
samples are fixed for each frame or a unit of frames. Such a modification would ease 
linear editing to prevent mismatches between audio and video data streams at a start 
point or an end point of an edited segment. 

Hashimoto et al., in combination with Kato and Riek et al., discloses the claimed 
invention except for multiplexing an audio and picture signal. MPEG-1 Part 1 teaches 
that it was known to produce a multimedia datastream by multiplexing packets of audio 
and video data. Therefore, it would have been obvious to one having ordinary skill in 
the art at the time the invention was made to implement the combination of a video and 
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audio file in Hasliimoto et al. as a multiplexing operation, as taught by MPEG-1 Part 1, 
since MPEG-1 Part 1 states in the Introduction that such a modification would allow for 
synchronized playback of audio and video data without having to buffer an entire sub- 
stream. 

Regarding claims 12 and 53, in Kato, "the JPEG standard is used in the still 
image compression and the MPEG standard is used in the moving image compression" 
(column 9: lines 12-14). 

Regarding claims 14 and 55, figure 12 of Hashimoto et al. shows video files and 
audio files stored in separate areas of memory card 16. 

Regarding claims 15 and 56, in Kato, as mentioned previously, video data is first 
stored in first memory 20 and then transferred to second memory 22 (column 3: lines 
54-63). This corresponds with writing multiplexed data to memory, reading the 
multiplexed data from memory, and recording multiplexed data on a recording medium. 
Additionally, in Kato, during the recording of a moving image, a still image from the 
sequence of moving images may be additionally transferred from the first memory to the 
second memory as an intra picture in an independent process of the moving picture 
recording (column 4: lines 1-10). This corresponds with encoding a video signal in the 
"first" encoding method, writing the signal to the memory, reading the signal from the 
memory, and recording the signal to the recording medium. 
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Regarding claims 16 and 57, figure 5 of Kato shows a compressed video 
encoder including DCT circuit 107, quantizing circuit 108, and variable length coding 
circuit 115 (column 5: line 60-column 6: line 24). 

Regarding claims 17 and 58, figure 12 of Hashimoto at al. illustrates audio and 
video files stored in the memory as having headers. 

Regarding claims 19 and 60, an MPEG-1 pack, containing 3 packets, is designed 
to have a pack rate of 29 Hz, or 1 frame per pack (Section 1-A.6.3). 

Regarding claims 20 and 61, Kato temporarily stores incoming data on first 
memory 20, and after re-encoding, permanently stores the data on second memory 22 
(column 4: lines 45-55, "the second memory 22 is the final storage medium"). 

Regarding claims 23 and 64, in Hashimoto et al., image data compression circuit 
12 may also perform image decoding (column 6: lines 62-66), and so corresponds with 
the claimed "video decoding means". The decoded video signals may be further 
processed in DSP 11 (column 6: lines 58-61), which outputs a video signal 26 to a 
display such as an LCD viewfinder (not shown in figure 8). This display corresponds 
with the claimed "displaying means". Digital audio signals may also be decoded in 
audio data compression/expansion circuit, transmitted to D/A converter, amplified and 
filtered in amplifier 2b, and output in output stream 26 to speaker 32 (column 5: lines 17- 
39). This corresponds with the claimed "audio outputting means". This process of 
reading data stored in memory card 16 (column 7: lines 34-50), like all other processes 
of the camera of Hashimoto et al., is controlled by CPU 23 (column 7: lines 15-16), 
which corresponds with the claimed "controlling means". 



Application/Control Number: 1 0/71 3,1 30 Page 1 2 

Art Unit: 2621 

Regarding claims 24 and 65, tine CCD in Hashimoto et al. has a resolution of 768 
X 480 pixels (column 6: line 44), and Kato inputs images at a resolution of 720 x 480 
pixels, in accordance with the NTSC standard (column 4: line 15), and produces an 
output of 320 X 240 pixels (column 4: line 24), in accordance with the GIF format. 
Although neither Hashimoto et al. nor Kato et al. record pictures at the VGA 640 x 480 
pixel standard, it would have been an obvious matter of design choice to modify the 
image sensing portion of camera of Hashimoto et al. or of Kato to produce 640 x 480 
pictures, since it has been held that a change in size of a component is generally 
recognized as being within the level of ordinary skill in the art. See In re Rose, 105 
USPQ 237 (CCPA1955). 

Regarding claim 49, in Kato, as shown in figure 1, system control circuit 26 
controls both the image compression circuit 18, first memory 20, and second memory 
22 (column 3: lines 41-63). This system control circuit corresponds with the claimed 
"controlling means". 

Regarding claims 50 and 67, in Hashimoto et al., incoming image data from a 
camera is processed in noise reduction circuit 10 and DSP 11 (column 6: lines 40-61), 
and incoming audio data from a microphone is processed in amplifier/filter 3a (column 
6: lines 18-21). 

Regarding claims 51 and 68, in Hashimoto et al., figure 14 illustrates the 
flowchart for transmitting and receiving data from the camera to an external device 
(column 10: line 41-column 11: line 42). Data from the memory card is transferred to 
FIFO 13 (column 11: lines 25-29), and transmitted to an external device via interface 



Application/Control Number: 1 0/71 3,1 30 Page 1 3 

Art Unit: 2621 

circuit 27 (column 7: lines 1-36). Like every other process in the camera of Hashimoto 
et a!., this process is controlled by CPU 23 (column 7: lines 15-16), which corresponds 
with the claimed "controlling means". 

Regarding claims 69 and 70, in Kato, digital signal processor circuit 14 
corresponds with the claimed "first picture encoder" that performs the step of receiving a 
picture signal. In Hashimoto et al., audio data compression circuit 3 corresponds with 
the claimed "second encoder" that performs the step of receiving an audio signal. In 
Kato, image compression circuit 18, as modified by Riek et al., corresponds with the 
claimed "picture generation" that performs the claimed step of "generating fixed data". 
In Hashimoto et al., FIFO 13, which combines audio files and image files (column 11: 
lines 43-61), corresponds with the claimed "third encoder" that performs the claimed 
step of "multiplexing". 

8. Claims 21, 22, 62, and 63 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Hashimoto et al. in view of Kato, Riek et al., and MPEG-1 as applied 
to claims 1 1 and 52 above, and further in view of US Patent 6,327,423 B1 (Ejima et al.). 
Claims 21, 22, 62, and 63 are directed to specific operations of causing a camera to 
perform an audio capture for a certain time. Hashimoto et al. teaches taking pictures 
when a shutter button is pressed (column 7: lines 20-24), Kato teaches operating a 
keyboard to issue image taking commands (column 3: lines 41-53), and Riek et al. 
teaches recording a still image while a still select button is depressed (column 4: lines 
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41-50). However, the above references do not teach operation for a time period to 
encode audio data. 

Ejima et al. teaches a camcorder that records sound data. Regarding claims 21 
and 62, figure 14 is a flowchart illustrating one embodiment of the sound recording 
control process of Ejima et al. At step SI, CPU 39 determines if a release switch 10 is 
pressed, and if It is, the image recording process begins at step S2 (column 15: line 64- 
column 16: line 3). At step S3, the sound recording process is started, and at step S4, a 
"REC" display Is shown on a viewfinder to indicate that sound is being recorded (column 
16: lines 4-11). At step S5, after 10 seconds have passed, the sound recording process 
stops (column 16: lines 11-16, 34-40). However, if a sound recording switch is pressed 
within 10 seconds at step S6, sound recording continues (column 16: lines 14-23, 44- 
50). The sound recording then ends when the sound recording switch is released at 
step S20 (column 16: lines 23-50). Then, sound recording switch 12 corresponds with 
the claimed "operating means", and the time period in which the sound recording switch 
is pressed corresponds with the claimed "timing means". 

Hashimoto et al., in combination with Kato, Riek et al., and MPEG-1, discloses 
the claimed invention except for encoding audio during the pressing of an operation 
means. Ejima et al. teaches that it was known to perform sound recording while a 
sound recording switch is pressed. Therefore, it would have been obvious to one 
having ordinary skill in the art at the time the invention was made to incorporate the 
sound recording switch of Ejima et al. into the camera of Hashimoto et al. or Kato, since 
Ejima et al. states in column 1 : line 60-column 2: line 20 that such a modification would 
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allow the tinning of a sound recording to be independent of the tinning of its associated 
video recording. 

Regarding clainns 22 and 63, in Kato, if sound recording switch 12 is not pressed, 
then release switch 10 corresponds with the clainned "operating nneans", and the ten 
seconds is the "predeternnined tinne period" in which audio is encoded. 

Conclusion 

9. This Office action is non-final due to the new supporting reference. 

10. The prior art nnade of record and not relied upon is considered pertinent to 
applicant's disclosure. "DVCPRO: A Connprehensive Fornnat Overview" {Uchida), 
published July 1996, mentions locked audio. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to David N. Werner whose telephone number is (571)272- 
9662. The examiner can normally be reached on Monday-Friday from 10:00-6:30. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Mehrdad Dastouri can be reached on (571) 272-7418. The fax phone 
number for the organization where this application or proceeding is assigned is 571- 
273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
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published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 



ID. N. W./ 

Examiner, Art Unit 2621 
/Dave Czekaj/ 

Primary Examiner, Art Unit 2621 



